Behavioral Cloning A Correction

We recently re p o rted on the application of a machinel e a rning (ML) technique to automated flight control using a simulated F-16 combat plane (Michie and Camacho 1994). Subsequent tests of our data-induced flying model have b roadly confirmed the re p o rt e d results but have also identified a lack of robustness. We had undere s t i m a t e d the latter and now re g a rd our re p o rt (Michie and Camacho 1994) as being, by omission, potentially misleading. Successes and shortcomings of behavioral cloning (l e a rning by imitation) have been reviewed by Urbancic and Bratko (1994). They discuss the following problem domains: p o l e a n d c a rt balancing (Michie, Bain, and Michie 1990; Chambers and Michie 1969), flight-simulator c o n t rol (Sammut et al. 1992), telephone-line scheduling (Kibira 1993), and crane-simulator control (Urbancic and Bratko 1994). Conclusions a re as follows: First, successful clones have been induced using standard ML techniques in all four domains. Second, the cleanup effect, whereby the clone surpasses its original, has been observed in all four domains. T h i rd, in all domains, the best clones were obtained when examples from a single human only were used. Fourth, the present approach lacks robustness in that there is no guarantee of inducing with high probability a successful clone from given data. Fifth, typically, the induced clones a re not sufficiently robust with respect to changes in the contro l task. Sixth, although the clones do provide some insight into the contro l c o m p romised if relevant agents remain blind to the craft’s curre n t bearing with respect to the ru n w a y ! The other weakness that stood out was the cursory nature of the definitions provided for signaling accomplishment of the goals of individual stages (“achievement goals” in the terminology of Laird and coworkers). T h e re is an evident connection between the two because it is precisely failure to relate sensed state variables explicitly to local goals that is lacking from pure learning by imitation. As a next step, one of us (R.C.) is refining the transition goals that signal completion of successive flight stages. The main tool for this purpose is a variant of inductive logic programming that supports the inductive learning of complex flight goals.